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1 Introduction 

{Sj ■ Modern communications networks, such as the Internet, are able at any time to share 

. their resources, for example bandwidth, among those calls or connections currently in 

, progress. Such calls may require simultaneous capacity from several resources in the 

network. For example, "streaming" applications require, for their duration, a reservation 
\ of bandwidth on each network link over which they connect. It may then happen that 

while each resource in the network, considered in isolation, has sufficient capacity to 
service the demand placed on it, the control of the network is such that the requirement 
for simultaneous availability of capacity ensures that over time demand cannot be met, 
and that the network is unstable, that is, that the number of calls present in the network 
tends to infinity. This is the phenomenon of entrainment and has been previously studied 
in this context by various authors, notably Bonald and Massoulie (2001), de Veciana et al 
(2001), Kelly and Williams (2004). In particular these authors have considered a broad 
class of so-called fair-sharing control strategies — see below. Bonald and Massoulie and de 
Veciana et al have shown that here the instability problems referred to above do not arise, 
i.e. that provided the various network resources individually have sufficient capacity, under 
any fair-sharing control the network will remain stable. In the present paper we study the 
phenomenon of entrainment in more detail and characterise some of those conditions which 
give rise to its occurrence. In particular we show that it frequently arises through poor 
control whenever there are no calls of certain classes in the network, and that only minor 
adjustments to control strategies are required in order to avoid it. Thus very flexible 
management schemes, including those utilising significant prioritisation, may be safely 
implemented. We further establish, in Section [3l some new results for the stability of 
Markov chains whose transitions rates possess a natural monotonicity property, yielding 
both necessary and sufficient conditions for the stability of a wide class of network controls. 
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Since network parameters — in particular call arrival rates — are unlikely to be known in 
advance, it is of particular concern to identify controls whose stability is robust with 
respect to variations in these parameters. 

We take as our model the following, which is essentially that introduced by Roberts and 
Massoulie (2000). Let R denote the finite set of possible call, or connection, types. We 
denote the state of the network at time t by n(t) = (n r (t), r £ R), where n r (t) is the 
number of calls of each type r in progress at that time. Calls of each type r £ R arrive 
at the network as a Poisson process with rate v r and have sizes which are exponentially 
distributed with mean /x" 1 . Arrival processes and call sizes are all independent. (As usual, 
the above distributional assumptions make for simplicity of analysis. However, it seems 
likely that all the results of this paper remain qualitatively correct for other distributions 
of inter arrival times and call sizes, subject only to these distributions having finite means, 
and to the same independence assumptions. ) When the state of the network is n, calls of 
each type r are allocated in total a bandwidth b r {n) > 0. We assume always that b r (n) = 
whenever n r = 0. The process n(-) is thus Markov with state space and transition 
rates given by, for all r £ R, 

{n + e r at rate v r , 
I 1 ) 
n — e r at rate ^ r b r (n), 

where e r denotes the vector whose rth component is 1 and whose other components are 
0. For each n define also b(n) = (b r (n), r £ R). We shall refer to b = (b(n), n £ Z + ') as 
the control for the Markov process n(-). The set of feasible controls b is defined by a set 
of capacity constraints 

^2a jr b r (n) < cj, j £ J, (2) 

r£R 

indexed in a finite set J, where each aj r £ {0, 1}. Here Cj may be thought of as the 
capacity of resource j, and a call of type r utilises resource j if and only if aj r = 1. 
In order to allow some results to be stated with sufficient generality, it is convenient to 
allow the possibility that, for any j, we may have Cj = oo (corresponding to the effective 
nonexistence of the resource constraint j). However, we assume, without loss of generality, 
that, for all r £ R, there exists some j £ J with Cj < oo and aj r = 1. 

Our interest is in identifying, for fixed values of the parameters u r , [i r , Cj, and aj r , r £ R, 
j £ J (which we shall regard as defining a given network) those controls b which are both 
feasible and stable, where we now take the latter to mean that the corresponding Markov 
process n(-) is positive recurrent. A control b for which n(-) is null recurrent or transient 
is referred to as unstable. 

For any feasible control b, for any n, we shall say that a resource j £ J is saturated if the 
corresponding constraint (|2]) is satisfied with equality. Following Bonald and Massoulie 
(2001), we shall further say that a feasible control b is Pareto efficient if, for all n and 
for all r such that n r > 0, there exists j £ J with aj r = 1 such that j is saturated (so 
that b r {ri) may not be increased without either decreasing b r i(n) for some r' / r or else 
violating the constraints ([2]).) It is sometimes helpful to consider controls which are not 
feasible, so we note in particular that the requirement of Pareto efficiency includes that of 
feasibility. 

For each r £ R we define K r = u r /[i r , which may be thought of as the rate at which 
"work" of type r arrives at the network. Many, but not all, stability results depend on 
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the parameters u r and \i r only through the corresponding K r . As observed by Bonald and 
Massoulie (2001), a necessary and sufficient condition for the existence of a stable feasible 
control is given by 

^ CLjrKr < Cj, j G J. (3) 

rS-R 

For the necessity of this condition, observe that if it is violated for some j, then since, for 
any feasible control b, we then have X^re-R a jr{^r — b r (n)) > 0, standard arguments — see, 
for example, Asmussen (2003, Chapter 1, Proposition 5.4) or the argument of Kelly and 
Williams (2004) — show that b cannot be stable. For the sufficiency of the condition ([3]), 
define, for any vector b = (b r , r G R), the corresponding complete partitioning control b 
by b r (n) = b r whenever n r > 0. Note that this defines a process n(-) which corresponds 
to | R | independent queues and is such that, in each queue r, arrivals occur at rate v r and 
departures (when n r > 0) at rate \x T b r . Thus a necessary and sufficient condition for the 
stability of this control is that b r > n r for all r G R. In particular we may choose b such 
that the corresponding complete partitioning control is stable and feasible if and only if 
the condition ([3]) holds. Further, given such a b, we may clearly define a Pareto efficient 
control b' such that b' r (n) > b r for all n and for all r such that n r > 0. The corresponding 
process n'(-) may then be coupled to the process n(-) corresponding to b in such a way 
that n' r {t) < h r {t) for all r and for all t > 0. Hence the condition (|3|) is also sufficient for 
the existence of some stable Pareto efficient control. 

It is also clear that, in the case \J\ = 1 of a single resource constraint, the condition ([3]) is 
sufficient to ensure the stability of any Pareto efficient control. This is not in general true 
when | J| > 1, as is shown by Example 11.11 b elow . which is a simplification of one given by 
Bonald and Massoulie (2001). 

Example 1.1. Suppose that R = {1,2}, J = {1,2} and that the matrix A = (aj r , j G 
J,r G R) is given by 
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Thus calls of type 1 are constrained by resource 1 only, while calls of type 2 are constrained 
by both resources 1 and 2. Suppose, without loss of generality, that C2 < c\. We assume 
the condition Q, which here becomes 

K\ + K 2 < Cl, K 2 < C 2 . (4) 

Consider the Pareto efficient control in which complete priority is given to calls of type 1. 
Then the long-run fraction of time in which the network is empty of calls of type 1 — and 
so resource 1 is available for use by calls of type 2 — is given by 1 — K\jc\. Since, when 
this resource is available, calls of type 2 are processed at rate C2, standard arguments for 
the stability of a single-server queue now show that the control is stable if and only if 
K 2 < 02(1 — Ki/c\). This is a condition which is generally more restrictive than Q above. 
When it is violated we have the phenomenon of entrainment referred to above; that is, 
from (J2J), each resource in the network, considered in isolation, has sufficient capacity, but 
the given Pareto efficient control is nevertheless unstable. 

In Example 11.11 ab ove instability may be considered as occurring because such high priority 
is given to calls of type 1 as to ensure that, when these are emptied from the system, 
resource 1 is thereafter consistently underutilised. As we shall show later, this problem 
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may in general be avoided by choosing Pareto efficient controls b such that, for each r, 
b r (n) is usually small whenever n r is small. In particular this property is possessed by 
the class of (weighted) fair-sharing controls introduced by Mo and Walrand (2000) in a 
generalisation of various classes considered earlier (see, for example, Kelly et al (1998)). 
These are defined as follows: for a > 0, a ^ 1, and weights w r , r € R, a weighted a- fair 
control, or bandwidth allocation, is given by taking, for each n, b(n) to maximise the 
concave function 

2^w r n r (5) 

subject to the capacity constraints (J2}. This class is further extended to each of the cases 
a = 0, 1, oo by taking the limit of the a- fair control as a tends to each of these values. (For 
a = 1 this is equivalent to replacing the quantity defined in © by J2 r eR w rn r log b r (n).) 
For the cases a = 1 and a = oo, De Veciana et al (2001) use Lyapunov function techniques 
to show that the condition ([3]) is sufficient for the stability of (weighted) a-fair controls. 
Bonald and Massoulie (2001) show that this result holds for general a by using fluid 
limits and appealing to a result of Dai (1995) for multiclass queueing networks. (In 
fact it is not certain that Dai's result is directly applicable to the present networks with 
their simultaneous resource requirements; however, Bonald and Massoulie's approach is 
essentially equivalent to showing that their function / defined by 

f( n ) = y2w rf i; i K- a -^— 

o a + 1 

reR 

is a suitable Lyapunov function for establishing the sufficiency of ([3j) for the stability of 
a-fair controls, and this requires only a small extension to their argument.) We note in 
particular that the stability of the fair-sharing controls is robust with respect to parameter 
variation, subject of course to ([3]) being satisfied. 

The resource underutilisation of Example 1 1 . 1 1 may be further understood as resulting from 
the nonsmooth nature of the call arrival process. Consider the analogous fluid model in 
which "work" of each type r arrives steadily at rate K r and is processed at rate b r (n), where 
each n r is now the volume of work of type r in the network and where b is again subject 
to constraints of the form (|2|). Then, under the condition © of Example 11.11 it is easy 
to see that every Pareto efficient control is stable, in the sense here that the total volume 
of work in the system eventually becomes and remains zero. For our stochastic model, 
the possible modified control discussed above, in which bi(n) is kept small whenever m is 
small and ni is large, may be seen as a smoothing operation forcing the behaviour of the 
stochastic model to follow more closely that of the fluid model. However, as Example 11.21 
below shows, instability may also occur in ways such that even the corresponding control 

for the analogous fluid model is also unstable. 

LRI \R\ 
For any control b and any function / on J/ + 1 , define the function D^f on Z,' + 1 by 

D h f(n) = Wr (f(n + e r ) - f(n)) + /i r 6 r (n) (f(n - e r ) - f(n))] . (6) 

reR 

(Since, for n and r such that n r = 0, we have also b r {n) = 0, there is no problem arising 
from the lack of a formal definition of /(n — e r ) in this case. Further, Df, may be thought 
of as the generator of the Markov process n(-) under the control b.) 
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Example 1.2. Consider the network denned by R = {1,2,3}, J = {1,2,3}, 

/0 1 l\ 

4=1 1 , 

v 1 °; 

with z/ r = is, jj, r = 1 for all r and Cj = c for all j. Assume that the condition ([3]) is satisfied, 
i.e. that 2v < c. Consider any control b such that, for all n / 0, we have b r (n) = c for 
some r such that n r > and 6 s (n) = for s 7^ r. Then b is Pareto efficient. However, 
for the function f(n) := ^reij 71 ^ anc ^ f° r an ^ 7^ 0, we have Dbf(n) = ?>v — c. Standard 
martingale arguments — again see Asmussen (2003, Chapter 1, Proposition 5.4) — now show 
that b is unstable whenever 3f > c. In this example the instability is not simply the result 
of poor control for n close to the boundary of lA, and is equally present in the analogous 
fluid model. We return to this example in Section [3l 

Our aim in the remainder of the paper is to identify more general classes of stable controls, 
to provide insight into how the phenomenon of entrainment rises, and to show how controls 
may be modified if necessary so as to avoid it. In Section Owe consider a number of fairly 
simple network topologies, and use Lyapunov function techniques to show that, for these, 
any Pareto efficient control b is stable, provided only that, as discussed above and for 
suitable r, b r (n) is modified so as to be small whenever n r is small. In Section [3] we 
consider controls which possess a natural monotonicity property, likely to be satisfied in 
any applications. We introduce some new analytical techniques to prove some fairly general 
results for the stability of the associated Markov chains. These give sufficient conditions 
for stability, which, for many classes of control, are also close to being necessary. These 
results are applicable to a substantial class of priority-based controls. 



2 Simple network topologies 

In this section we consider a number of network topologies and in each case show that, 
under the condition ([3]), any Pareto efficient control is stable provided only that it is 
suitably modified for values of n close to the boundary of Ij + . 

The results of this section are based on simple Lyapunov function techniques, in particular 
Foster's criterion. Proposition 12.11 below states the specialisation of this to the present 
problem — see, for example, Asmussen (2003, Chapter 1, Proposition 5.3(h)), noting that 
here jumps of the process n(-) may only occur between neighbouring states, and also that 
the usual uniformisation argument translates statements for discrete-time processes to the 
present continuous time setting. 

Proposition 2.1. Given any control b, suppose that there exists a positive function f on 

\R\ \R\ 
Z + , a finite subset F ofZ + and some e > such that Dbf(n) < —e for all n ^ F. Then 

b is stable. 

In considering the various network topologies of this section, it is convenient to define, for 
any a > 1, the function g a on Z + by 

{a n 2 
- + — , if n < a, 
2 2a 
n, if n > a, 
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We first consider again the very simple Example 1 1.1 1 of the Introduction, but with a general 
control b. 

Example 2.1. Let R, J and A be as in Example 11.11 and assume again that c 2 < c\ and 
also the condition (J3J). We show that, given any 5 > 0, there is some a > 1 (depending 
on the parameters u r , fj, r and Cj and 6) such that a sufficient condition for the stability of 
any Pareto efficient control b is given by 

K2 — b2(n) < —5, whenever n\ < a and n ^ F, (7) 

for some finite set F (necessarily including {n: n\ < a,ri2 = 0}). (Note that, from 
condition (J7|) may be satisfied for any 5 £ (0, c 2 — k 2 \.) 

Observe first that, from @ and the saturation of the resource 1 whenever there 
exists 5' > such that, for any Pareto efficient control b, 

2 

J2(K r - b r {n)) < -5', for all n ^ 0. (8) 

r=l 

For any fixed a, define the function f a on by 

, , s g a {ni) n 2 
Jain) = 1 . 

Ml fJ-2 

Elementary calculations show that, from (J6j) , for any Pareto efficient control b satisfying 
condition (JT]) for some finite F, and for any n ^ F, 

Dbfain) = min (—, l) [ki - h(n)] + k 2 - b 2 {n) + ^-h 1>a (n) 
v a J la 

< - min ( — , l) 5' - max f 1 - — , o) 6 + —h x a (n) (9) 
v a J \ a ) 2a 

<-mm(5,5 f ) + ^h lta (n), (10) 
la 



where the function h\ A is given by 

hi, a (n) = 



K\ + bi(n), if < rii < a, 

h(n), \in x = a, (11) 

0, if n\ > a, 



and where Q follows from ([7|) and (|H|). Since bi(n) < c\ for any feasible control b, it 
follows from (jlOf) and (llip that a may be chosen sufficiently large that, for any Pareto 
efficient control b satisfying (|7|) for some finite F, 

D b f a (n) <~ mm(<5, 5') for all n(£F, 
and so, by Proposition I2.1|, b is stable. 

Thus, for this example, any Pareto efficient control is stable provided only that, outside 
of some finite set F, it is suitably modified for values of n such that n\ is small. However, 
in practice the parameters v r in particular are unlikely to be known, and so it is desirable 
to choose controls whose stability is robust. One such possibility is to choose any Pareto 
efficient control which assigns complete priority to calls of type 2 whenever n\ < a for 
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some a. Provided only that the condition ^ is satisfied, the above result shows that this 
will be stable provided a is sufficiently large. The precise value of a required depends on 
the slack in the inequalities (jlj), but calculations for the "worst case", in which calls of 
type 1 have complete priority whenever n\ > a, show that in general a need only be small. 

Note in particular that the various fair-sharing controls defined in the Introduction always 
satisfy the condition ([7]) for some 5 and for some sufficiently large F, and hence (as already 
remarked) are always stable provided that the condition (|3J) is satisfied. 

We now extend the above example to each of two more general network topologies. In 
each case it is again the case that only small modifications, identified below, are required 
to Pareto efficient controls in order to ensure their stability. 

Example 2.2. Consider the network with R = {1, . . . ,\R\}, and in which each call type r 
requires service from a dedicated resource of capacity c r together with service from a 
resource which is shared by all call types and has capacity cq. The constraints ([2]) defining 
the feasible controls b are thus 

^6 r (n)<c , (12) 

r€R 

6p(n) < Cr, rG R. (13) 
We assume, without loss of generality, that Co < oo and that 

c <J2 c r- ( 14 ) 
As usual we assume the condition (jSJ), which here becomes 

^2 Kr < c °' ( 15 ) 

r<=R 

K r < c,., r G R. (16) 

The case \R\ = 2 with c\ = oo is the earlier Example 12.11 (with cq here corresponding to 
ci there). However, in the general case described above a little more care is needed in the 
conditions for the stability of Pareto efficient controls. 

For any n £ J} + , define n m j n = min rg ^ n r . We show briefly that, given any 5 > 0, there is 
again some a > 1 (depending on the parameters v r , [x T and Cj and 5) such that a sufficient 
condition for the stability of any Pareto efficient control b is given by 

(n r — b r (n)) < —5, for all a' £ [1, a] and n such that n m i n < a, n ^ F, (17) 

r : n r >a' 

for some finite set F. (The existence of a 5 > such that the condition (|17p may be 
satisfied is guaranteed by (fT5l) and (fT6l) . Further, it is not difficult to see that in the 
case R = 2, this condition reduces to that of Example 12.11 taken together with a similar 
condition in which the roles of calls of types 1 and 2 are interchanged.) 
The proof of this result is similar to that of Example l2.1i Note first that it follows from (|14p 
that, for any Pareto efficient control, the resource is necessarily saturated for any n such 
that n m i n > 1. Hence, from (fTf)|h there exists 8' > such that, again for any Pareto 
efficient control b, 

^^(ft r — b r (n)) < —5' for all n such that n m j n > 1. (18) 

rG-R 
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\R\ 

For any fixed a, define the function f a on ZL_ 1 by 



fa(n) = 



9a\n r 



Elementary calculations show that, from ©, for any Pareto efficient control b satisfying 
the condition (|17f) for some finite F, and for any n £ F, 

Dbfa{n) = ^ min (~> X ) K ~ M n )] + ^ h rA n r)-> 
reR r£R 
l a 1 

= - V] (/c r - b r (n)) + — y~] h r a (n r ) 

a'=l r: n r >a' r£R 

< -min(S,5') + — J2 h rAn r ), (19) 

r€R 

where, for each r, the function h r>a is given by with the index r replacing the index 1, 
and where (|19j) follows from (|17j) for n such that n m i n < a and from (|18|) for n such that 
^min > o- It now follows as in Example 12.11 that a may be chosen sufficiently large that, 
for any Pareto efficient control b satisfying (|17p for some finite F, 

D b f a (n) <-- min(5, 5') for all n £ F, 
and so, again by Proposition 12, 1\ the control b is stable. 

Thus, in order to ensure the stability of a general Pareto efficient control b, it is only 
necessary to appropriately modify b(n) for n such that n m i n < a. In particular it follows 
from the above result that a Pareto efficient control b whose stability is reasonably robust 
is given by requiring that, for some a, for all n (such that n m \ a < a) and for all a' £ [1, a], 
calls of types r such that n r > a' collectively have complete priority over calls of the 
remaining types, that is, that 

b r (n) = mini Co, c r | . 

r: n r >a' r: n r >a' 

For a sufficiently large, depending on the slack in the inequalities (]15p and (|16p . any such 
control is stable. 

Now consider further the case R = 2. Note that the present topology is completely general 
for a network with two call types. Let b be any Pareto efficient control such that 

lim &i(ni,ra2) = ci for all n-z, (20) 

ni^oo 

lim 62(^,712) = C2 for all m. (21) 

ri2— >oo 

Then it is straightforward that, for 5 < min r =i,2( c r — K r) 5 and for any a, the condition (|17p 
is satisfied for F sufficiently large, and so b is stable. 

In particular the conditions (|20p and (|2ip are again satisfied by the various fair-sharing 
controls. The present conditions are of course considerably more general. However the de- 
velopment of corresponding results for general networks with R > 3 remains a challenging 
problem. 
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Our final network topology is a simple "backbone" structure. 

Example 2.3. Consider a network with resource set J = {1, . . . , k}, where resource j has 
capacity Cj as usual. The set of call types is given by R = {0,1, ... ,k} where calls of 
each type r = 1, . . . , k require service from the single resource j = r, while calls of type 
require service from each of the resources 1, . . . , k. 

The constraints ([2]) defining the feasible controls b are given by 

6 (n) + b r (n) < c r , r = l,...k. (22) 

Again we assume the condition (|3j), which here becomes 

Ko + K r < c r , r = 1, . . . k. (23) 

The state of the network is thus here denoted by n = (no,n\, . . . , re^.). For any such n 
define n max = max(m, ... ,n k ). 

This example again generalises that of Example 12.11 with the call type here playing the 
role of the call type 2 in that example. In a fairly straightforward generalisation of the 
result of that example, it is here the case that, given any 5 > 0, there is again an a > 1 
such that a sufficient condition for the stability of any Pareto efficient control b is given 
by 

Ko — bo(n) < —5, whenever n max < a and n ^ F, (24) 
for some finite set F 

To see this, suppose first that /i r = 1 for all r = 0, 1, . . . , k and that the condition (j2l"|) is 
satisfied. For any a > 1, define the function f a on by f a (ri) — tiq ~\~ Qaij^max) ■ Then, as 
in the case of Example 12.11 it follows straightforwardly from the conditions f|23|) and (|24[) 
that there exists 5' > and a sufficiently large that Dbf a (n) < —5' for all n £ \ F 
such that additionally n max = n r for a single value of r = 1, . . . , k. It is now easy to see 
that this latter restriction may be removed (possibly at the expense of increasing a) by 
suitably smoothing the function / in the neighbourhood of those ti such that fi max — n r 
for two or more values of r = 1, . . . , k. The desired result thus follows in this case; for 
general fi r only routine modifications to the above argument are required. 

Hence we again have that any Pareto efficient control requires only slight modification — 
for those n such that n max < a — in order to be stable. A robust Pareto efficient control is 
given, for example, by assigning complete priority to calls of type whenever n max < a, 
the necessary value of a depending on the slack in the inequalities (I23p . 

Examples I2.1H2.3I above all consider fairly simple network topologies. In the analogous 
fluid model defined in the Introduction, it is easily seen that, for each of these topologies, 
condition ([3]) is sufficient for the stability of any Pareto efficient control. (In each case this 
follows, for example, by using the same Lyapunov function as for the stochastic model, 
except that the function g a may be replaced by the identity function.) The examples 
illustrate a principle which seems likely to be true for more general network topologies, 
namely that when a control is such that it is stable for the fluid model, then there is a 
closely approximating control which is stable for the corresponding stochastic model. 

In the next section we consider stability criteria for quite general network topologies, 
applicable typically to controls where there is some prioritisation among call types. 
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3 Stability of monotonic controls 



Many controls likely to be of practical application possess a simple monotonicity property 
(see below). In this section we study stability for a wide class of such controls, giving 
sufficient conditions for stability, which, for many classes of control, are also close to being 
necessary (see Remark [32] below which further discusses the applicability of the results of 
this section). We require first the following quite general lemma. 

We shall say that a control b is bounded if, for all r, b r (n) is bounded in n. 

Lemma 3.1. Let b be any bounded control and, as usual, let n(-) denote the Markov 
process for the corresponding network. Then, for all r £ R, 

1 f l 

limsup- / b r {n{u)) du < K r a.s. (25) 

t^OO t Jq 

Proof. For all r £ R, the compensated process ?%*(•) defined by 

n*(t) = n r (t) — n r .(0) + / (b r (n(u)) — K r ) du 



is a zero-mean martingale. Since also the transition rates of the Markov process n(-) are 
bounded, it follows that, for some constant M and for all t > 0, we have B(n*(t) 2 ) < Mt. 
Thus, for 1/2 < a < 1, the process (n*(t)/t a ) t>0 is an £ 2 -bounded supermartingale, and 
so, as t — > oo, converges almost surely to some finite random variable. It follows that 

n r (t) 1 



i r* 

H — / b r (n(u)) du —> K r a.s. 
t Jo 



t 

Since the process n. r (-) is positive, the result ([23]) now follows. □ 

Remark 3.1. In the case where the bounded control b is stable, the Markov process n(-) 
is positive recurrent, and so we have the stronger result that, for all r G R, 

1 f* 

lim — / b r (n(u)) du = Effbr = K r a.s. (26) 
t-*oo t J 

Here 'Ei^br denotes the expectation of the function b r with respect to the stationary dis- 
tribution 7r of n(-), the first equality in (]26[) follows from the ergodic theorem, and the 
second is simply the assertion that, under stationarity (and readily deducible from the 
balance equations defining 7r), the expected arrival and departure rates for calls of type r 
are equal. 

We shall say that a bounded control b is monotonic if, for all r and for all n, 

b r (n) is increasing in n r (with n s fixed for all s ^ r), (27) 
b r (n) is decreasing in n s (with n s i fixed for all s 7^ s), for all s ^ r. (28) 

Note that, depending on the network structure, this property is natural in many applica- 
tions. For instance, for the structure of Example 12.21 it is possessed by all the fair-sharing 
controls, and also by any other reasonable Pareto efficient control. For more complex 
network structures, controls may be coupled to monotonic controls to establish stability 
results using the results given below. See also Example 13. II below. We note further that a 
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related but somewhat different definition of monotonicity is used by Bonald and Proutiere 
(2004). 

For any monotonic control fo, and for each S C R (including the case where S is the empty 
set 0), define the function b s : z!f' -> M^ 1 by 



6f (n s ) = lim b r (n), r G 5, (29) 

n s — >oo V 



&f(n s ) = lim lim b r (n), r 4 S, (30) 

n r ^oo n s — >ooVs^5U{r} 



where = (n s , s E S) and where, in taking the limits in the right side of each of 
and (|30p . the vector of those coordinates of n that belong to S is held fixed at ng. Note 
that, by monotonicity, this function is well-defined. In particular, in (j29H . the order within 
R \ S in which the limits are taken is irrelevant; however, in (I30p . the final limit to be 
taken must be that as n r — ► oo. 

For any monotonic control 5 and any S 1 as above, we shall say that b s is stable if the 
application of the control (6f , s £ 5) to calls in the network of types s G 5 (with z/ s , 
s G 5, as usual) yields a positive recurrent Markov process n (•) = (nf (•), s G 5). (This 
| SI -dimensional process may be thought of as that which results when the number of calls 
of each type r ^ S is infinite.) When b s is stable we shall let ir s denote the stationary 
distribution of n (•) (or, where necessary, the probability function of this distribution); 
we shall further define, for each r € R, 

E„sb?= K s (n s )b s r (n s ) (31) 

to be the expected value of b^ under this distribution. In the case where S is the empty 
set 0, we have that = (bf, r € R) is a vector of constants. We make the natural 
convention that b® is always stable; the distribution ir® is concentrated on a single point, 
and we have E^e^f = bf, for all r G R. 

Our main result of this section is Theorem 13.11 below. The first part is similar in spirit to 
results of Borovkov (1998, Chapter 8) for asymptotically spatially homogeneous Markov 
processes. However, the application of those results here would require that the right side 
of (|30p is invariant under interchange of the limits in that expression, a condition which is 
not in general satisfied for our monotonic controls. Rather the monotonicity itself provides 
sufficient structure to obtain the results of Theorem 13.11 

Theorem 3.1. Suppose that the control b is monotonic and that S C R is such that b s 
is stable. 



(i) If r £ S is such that 

then & 5LJ M is stable, 
(ii) If r $l S is such that 

then b SlJ { r } is unstable. 



B n sb^ > Kr, (32) 



E^s&f < Kr, (33) 
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Remark 3.2. Given the stability of b s for some S C R (recall that, as already remarked, b® 
is always stable) , Theorem 13.11 gives criteria for determining the stability or otherwise of 
6 SU W for any r £ S, except only in the case of equality in ([32]) or ([33]) (where the natural 
conjecture is that b Su ^ is unstable — see also the remarks at the end of Example 13. ip . 
Recursive application of the theorem thus yields sufficient conditions both for the stability 
and the instability of monotonic controls. However, note that, for example in the case 
R = {1,2}, &« and &W may both be unstable, while b = b^- 1 ' 2 ^ is stable, as is the 
case for fair-sharing controls here. In such circumstances Theorem 13 . 1 1 does not settle the 
question of the stability of b. Rather its primary application is to controls in which there 
is a sufficient hierarchy of prioritisation as to permit the recursive application of the first 
part of the theorem to at least establish the stability of b s for S = R\ {r} for some r G R. 
The theorem then also (in general) settles the question of the stability of b itself. For an 
illustration of the application of the theorem, see Example 13.11 below. 

Proof of Theorem VJ.1\ Since, for given S C R and r ^ S, the stability of b Su{r} corre- 
sponds to the positive recurrence of the Markov process n Su ^(-) defined above (in which 
the number of calls of each type s ^ 5u{r} is effectively held at infinity), it is sufficient to 
prove the results (i) and (ii) in the case where S = R\{r'} for some r', and we henceforth as- 
sume this. (The primary advantage of doing so is that we avoid some unpleasant notational 
complexity.) We identify any n G 1} + with the pair (ns,n r >) where ns = (n s , s G S). 
Recall that then, for each such ns, we have bf,(ns) = lim n b r >(ns, n r i). 
Suppose first that the condition ([32]) holds. We require to show that b is stable. The 
underlying idea is that the monotonicity of b and stability of b s ensure that the components 
(n s (-), s G S) of the process n(-) become and remain small, and the condition (|32|) then 
ensures that, except in some finite region A, the remaining component n r i(-) of this process 
is decreasing at a rate which is bounded away from zero; thus the process n(-) spends, in 
the long term, a nonzero proportion of time within A. 

It follows from (|32p and the monotonicity of b that we can choose a finite set A = {n : n r < 
n r , r G R} C z!^ and a positive function b r > on such that 

b r > is decreasing in each of its arguments, (34) 
^2 7r 5 (n s )6 r /(n s ) > n r >, (35) 

b r '(n) > b r i(ns) for all n £ A. (36) 

For example, given n r , r 6 R, we may choose 

b r >(n s ) = I(n s < n s , s G S)b r >(n s ,n r >), n s G zf ', (37) 

where I denotes the indicator function. The condition (I34p then follows from the mono- 
tonicity of b; since also b is nonnegative the condition (I36p follows trivially from (|37|) . 
except for n such that n s < n s for s G S and n r > > n r /, in which case (|36p again follows 
from the monotonicity of b; finally the condition (|35p follows from (|32p and the monotone 
convergence theorem by choosing n r , r G R, all sufficiently large, since, for any ng, 

lim I(n s < fi s , s G S)b r /(ns,n r >) = lim b r >(ns,n r r ) = b^,(ns)- 
n r — «3oVr6B n r '~ *°° 

It also follows from the monotonicity of the control b that we can couple the corresponding 
Markov process n(-) to a process n s (-) = (nf(-), s G S) on zlf with control b s in such 
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a way that n s (t) < n~ (t) for all i > and for all s G S. Since also the process n °{-) has 
stationary distribution tt s , it follows from (|34f) that, for the function b r i defined above, 

1 /"* - 1 /"* - 

limsup- / b r '(ns(u)) du > lim - / b r r(n s (u)) du = E^siv, (38) 

where ng(-) = ■? G S) and where the final equality above follows by the ergodic 

theorem. 

It now follows from Lemma 13. II that, for some M > 0, 

1 f* 

k t i > limsup— / b r i(n(u))du 

t^oo t Jq 

1 /•* 

> limsup- / b r i(n(u)) I((n(u) ^ A) dit 

t^oo i jo 

1 /*- 

> limsup- / 6 r /(n s (u))I((n(n) ^ A) dit (39) 

t^oo t Jq 

If 1 - M f* 

> limsup- / b r i(ns{u)) du — lim — / I((n(u) £ A) du (40) 

t-^oo t Jq t^co t Jq 

M f* 

> E^sK, - lim — / I((n(n) G A) dtt, (41) 

t-KX t J 



where the inequality (I39p follows from (|36|) . the inequality (I40p follows since, from ([3i 
the function 6 r / is necessarily bounded (note that, since A is finite, the limit in the final 
term in (I40p always exists), and the final inequality (I4ip follows from (1381) . Thus, from 
(35) and (E5D, 

1 /** 

lim - / I((n(n) G A) du > 0. 

t^oo t J 

Since A is finite it now follows from the ergodic theorem that the Markov process n(-) is 
positive recurrent and so b is stable. 

Now suppose instead that the condition (|33p holds. We show that the Markov process n(-) 
corresponding to b is transient (and hence b is unstable) . The underlying idea here is that 
whenever n r /(-) is very large, the process n(-) again behaves approximately as if it were 
controlled by b s , and thus, from (|33|) . we may expect that lim^oo n r i(t) = oo a.s. To make 
this rigorous we again use the monotonicity of b to couple the process n(-) to a process n(-) 

whose control is sufficiently close to that of b that we may show that lini^ >00 n r '(t) = oo 

a.s., and for which the coupling ensures that also limt_»oo n r '(t) = oo with strictly positive 
probability. 

Given n r > G Z + (fixed, to be chosen later), define a Markov process n(-) = (n r (-), r G R) 
as follows: for each s G S, the component process n s (-) has state space Z + as usual, while 
h r >{-) has state space Z; for each r G R, transitions n r — > n r + 1 occur at rate f r as usual, 
while transitions n r — > n r — 1 occur at rate fj, r b r (n), where 

b s (n) = b s (ns,n r >), s e S, (42) 
b r r(n) = b?,(n s ). (43) 

Observe that the process n(-) has uniformly bounded transition rates which are indepen- 
dent of n r i G Z. Suppose that 

n(0) = n(0), n r /(0) > n r '- (44) 
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Define the random time T = min{t > 0: n r i(t) < n r i}. It follows from the monotonicity 
of b that, for all n such that n r i > n r >, 

b s (n) > b s (n) > b s s (n s ), s E S. (45) 

and hence that we may couple the processes n(-), n(-) and the jS 1 (-dimensional pro- 
cess n s (-) with control (fr~, s G S) in such a way that 

n s (t) < nf (t), for all s G 5 and for all t > (46) 

and 

n s (t) <n s (t), for all s G 5 and for all < f < T. (47) 

Since n s (-) is assumed positive recurrent with stationary distribution tt s , it follows from 
([431) or (|4T>j) that n,s(-) = (n s (-), s £ 5) is similarly positive recurrent with stationary 
distribution ns say. Further, as n r > — > oo, the control b converges pointwise in each of its 
components to b s . Hence elementary arguments (e.g. consideration of the times of return 
to of the process n s {-), coupled with the use of (|46p and the ergodic theorem) show 
that, again as n r i — > oo, ns converges in distribution to ir s . Since also bf t is bounded, it 
now follows from (|33|) that we may choose the constant n T i sufficiently large that 

E^ s 6f, < K r >, (48) 

(where, analogously to (f3T1) . E^- s 6^, is the expectation of bf, with respect to tts). 
It further follows from the monotonicity of b and from (|47h that, for all < t < T, 

b r ,{h{t)) = bS(n s {t)) 

> b S r ,(n s (t)) (49) 

> b r ,(n(t)), (50) 

where (|49p follows from (|47p and the monotonicity of b while (|50p follows from the defini- 
tion of b s and, again, the monotonicity of b. It now follows from (|50p that we may couple 
also the components r' of the processes n(-) and n(-) in such a way that 

h r >(t) < n r ,(t), for all < t < T. (51) 

As noted above, the process fis(-) = (n s (-), s G S) has stationary distribution tts, while 
the process rv(-) may be viewed as a Markov additive process modulated by the remaining 
components ns(') of n(-). From (|48p . the expectation of the increments of h r >(-) between 
those times at which h(-) returns to any fixed state is strictly positive. It follows from the 
standard theory of Markov additive processes that lim^oo n r > (t) = oo a.s., and further 
that, under the condition (f4*4"j) . 

P(h r /(t) > n r i for all t > 0, lim h r '(t) = oo) > 0, 

t— >oo 

and hence, from (I5ip . that also 

P(n r /(t) > n r i for all t > 0, lim n r /(t) = oo) > 0. 

t^oo 

Hence the process n(-) is transient as required. □ 
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We illustrate the use of the above result with a simple example. 

Example 3.1. Consider again the network of Example 11.21 i n which R = J = 3. As 
previously observed a necessary and sufficient condition for the existence of some stable 
control is given by 2u < c. Further, if 3v < c, then Proposition 12.11 with the Lyapunov 
function / given by f(n) = Yl r =i n r shows that any Pareto efficient control is stable. 
Suppose now that 2v < c and that the Pareto efficient control b is such that, for r = 1, 2, 
b r (n) is independent of 77.3 and 

b\{n) + b 2 (n) = c for all n such that max(ni,rt2) > 0. (52) 

Thus in particular calls of types 1 and 2 collectively have complete priority over calls of 
type 3. Although we do not in this example require any further monotonicity conditions 
on b, it follows from the requirement of Pareto efficiency that the control b^ 1,2 ^ : Z 2 ^ — > 
is well-defined as before, being obtained from b by letting 713 — ► 00. It follows from (|52[) 
that the condition 2v < c is necessary and sufficient to ensure that 6 {1 ' 2} is stable. We 
use (a slight modification of) Theorem 13. II to investigate the stability of b. The stationary 
distribution T^ 1 ' 2 } on induced by b' 1,2 } is here just that of the process (ni(-), n 2 (-)). 
Further, since, from (j52H . ni(-) + 7i2(0 is Markov, with a stationary distribution which is 
geometric and independent of any more detailed specification of fo^ 1 ' 2 -^, it follows that 

^ 2 >(0,0) = 1--. (53) 
c 

It follows from ([2]), (j52H and the Pareto efficiency of b that 

4 1,2} (°' °) = c > 4 1,2} (ni,0) = forallni>l, 6j 1,2} (0, n 2 ) = for all n 2 > 1 

(54) 

We thus have that 

(ni,n 2 )GZ5j_ 

> c - 2u, (55) 

with equality if and only if 63 ' {n\,n 2 ) = for all (711,712) such that min(ni,?i2) > 1. 
But this latter condition holds if and only if, for all (7ii,n 2 ) such that min(ni,n2) > 1, 
we have mia(&| 1,2 ^(ni, n 2 ), b^"^ {n\, n 2 )^ = 0, i.e. in the case of the control considered in 
Example 11.21 in which maximum resource is always allocated to calls of one type, and in 
which we have already observed that we have stability if and only if ?>v < c. Otherwise 
we have strict inequality in (j55f) . 

Now note that, although b does not here satisfy all the conditions for monotonicity given 
earlier, the assumption that b\ (n) and b 2 (n) are independent of 713 ensures that Theo- 
rem 13.11 continues to apply, indeed in a slightly improved form, to show that the condi- 

M 2 \ 

tion E {1,2} 63 > v is necessary and sufficient for the stability of b. (For the sufficiency, 
note that the proof of part (i) of the theorem, with S = {1,2} and r' = 3, goes through 
as before, except that the coupling between n(-) and process n s (-) is now obtained with 
equality, and so we no longer require the condition (|34p in order to obtain (|38p . Similar 
obvious simplifications apply to the proof of part (ii), which here becomes a fairly stan- 
dard argument and in particular delivers null recurrence — and hence instability — in the 
case ^{1,2} 03 = v.) 
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Suppose now that c and b are held fixed and that v is allowed to vary. The obvious 
coupling argument shows that if b is stable for any v then it is also stable for any v' < v. 
The above adaptation of Theorem l3,ll together with (|55H . shows that there is some critical 
parameter A (depending on the detailed specification of b^ 1,2 ^ and hence tt^- 1 ' 2 ^) such that 
1/3 < A < 1/2 and that b is stable if v < Ac and unstable if v > Ac. For the control of 
Example 11.21 we already know that A = 1/3; otherwise for the case v = c/3 we have strict 
inequality in ([55]) and hence stability; simple continuity arguments now give A > 1/3 in 
this case. 
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